Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Scala.JS linker in an external process, enable back Scala.JS linking on Windows #676

Merged

Conversation

alexarchambault
Copy link
Contributor

No description provided.

@alexarchambault alexarchambault mentioned this pull request Feb 21, 2022
@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from d6a08fc to 810294a Compare February 22, 2022 10:00
@alexarchambault alexarchambault marked this pull request as draft March 9, 2022 11:06
@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from 810294a to 22f99af Compare March 15, 2022 13:08
@alexarchambault

This comment was marked as outdated.

@alexarchambault

This comment was marked as outdated.

@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from 22f99af to 709f3c7 Compare March 17, 2022 16:38
@alexarchambault

This comment was marked as outdated.

@alexarchambault

This comment was marked as outdated.

@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from 709f3c7 to 16a42db Compare March 21, 2022 15:04
@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch 4 times, most recently from 1b6b3e3 to aab6047 Compare March 22, 2022 15:27
@alexarchambault alexarchambault changed the title Enable back Scala.JS linking on Windows Run Scala.JS linker in an external process, enable back Scala.JS linking on Windows Mar 22, 2022
@alexarchambault alexarchambault marked this pull request as ready for review March 22, 2022 15:30
We don't write .scala-build / .bsp under modules anymore, no need of
these in the root .gitignore
@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from aab6047 to 1cc72f5 Compare March 23, 2022 09:00
@sjrd
Copy link

sjrd commented Mar 23, 2022

I'm a bit worried that you have to resort to using scalajs-cli in scala-cli. scalajs-cli should be a last resort. It is always better to use the linker API if one can do so. Reasons include:

  • scalajs-cli incurs the JVM startup time
  • it cannot reuse its Linker instance, and therefore cannot work incrementally.
  • it does not support all the linker configuration operations
  • it is barely maintained

So that we don't have to embed the whole scala-js linker in the Scala
CLI native image.
@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from 1cc72f5 to e7a65d3 Compare March 23, 2022 09:35
@alexarchambault
Copy link
Contributor Author

  • scalajs-cli incurs the JVM startup time

We default to native launchers of scala-js-cli in the current state of this PR. These are built from this repository and pushed as GitHub release assets, where Scala CLI later downloads them. (We already do that kind of thing for scalafmt and metabrowse).

  • it cannot reuse its Linker instance, and therefore cannot work incrementally.

Not sure we were using that from Scala CLI for now anyway. I guess we'll have to make scala-js-cli a server to use that.

  • it does not support all the linker configuration operations

In the current state of this PR, we rely on this fork of scala-js-cli (while waiting for scala-js/scala-js-cli#42). I guess it's fine for us to add missing features to scala-js-cli as we add support for them in Scala CLI.

  • it is barely maintained

We can basically vendor it for Scala CLI if needed (via this fork). From the Scala CLI point of view, it's basically bits of code that used to be in Scala CLI, and now live in a separate app.

@alexarchambault
Copy link
Contributor Author

alexarchambault commented Mar 23, 2022

@sjrd One of the motivations for that is to save the cost of processing the Scala.JS linker classes when generating Scala CLI native images. We're a bit short on memory on our CI workers, to the point that it prevents us from merging some newer developments. And the Scala.JS linker classes seem to take up quite some space in the Scala CLI native images.

@alexarchambault alexarchambault force-pushed the scala-js-linking-windows branch from e7a65d3 to cc4e896 Compare March 23, 2022 12:20
@alexarchambault alexarchambault merged commit a5d7bd7 into VirtusLab:main Mar 23, 2022
@alexarchambault
Copy link
Contributor Author

Seems native image generation works much better with these changes: it takes less than 10 minutes on Linux and Windows, none of the native image jobs complains about time spent in GC, and the native-image memory logging doesn't go above ~3.5 GB 🎉

@alexarchambault alexarchambault deleted the scala-js-linking-windows branch March 23, 2022 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants